Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bump monty to use the monty.json import speedup patch, add import time regression test, lazy load some rarely used but costly modules #4128

Merged
merged 49 commits into from
Nov 13, 2024
Merged

bump monty to use the monty.json import speedup patch, add import time regression test, lazy load some rarely used but costly modules #4128

merged 49 commits into from
Nov 13, 2024

Conversation

DanielYang59
Copy link
Contributor

@DanielYang59 DanielYang59 commented Oct 22, 2024 •
edited
Loading

Summary

  • bump monty to use the latest json import speedup patch, partially fix import monty.json slowing down core import #3793

  • Have a quick look at other important modules (on cover those import other 3rd-party/non-core-pmg modules in this PR) and add import time test (profile: python -X importtime -c "from pymatgen.core.structure import Structure" 2> pmg.log && tuna pmg.log):

    • core.bonds: image
    • core.composition: image
    • core.interface (need attention)
    • core.ion: image
    • core.lattice: image
    • core.operations: image
    • core.periodic_table: image
    • core.sites: image
    • core.spectrum: image
    • core.structure
    • core.surface: image
    • core.tensors: image
    • core.trajectory (AseAtomsAdaptor lazy imported)
    • io.vasp.inputs image
    • io.vasp.outputs: image

  • [For a follow up PR] looks like scipy across core need special attention

@DanielYang59 DanielYang59 changed the title bump monty to use the latest json import speedup patch bump monty to use the latest monty.json import speedup patch Oct 22, 2024
@DanielYang59 DanielYang59 mentioned this pull request Oct 22, 2024
Merged
@DanielYang59

This comment was marked as resolved.

Sign in to view
@DanielYang59 DanielYang59 force-pushed the bump-monty-json branch 4 times, most recently from 5489dff to 8d9d9a6 Compare October 22, 2024 06:54
@DanielYang59
Copy link
Contributor Author

DanielYang59 commented Oct 22, 2024 •
edited
Loading

@mkhorton Can I lazy import sympy (it's used by only one method symmetry.settings.JonesFaithfulTransformation.parse_transformation_string and would give us a ~15% speed up on import core.Structure)?

@DanielYang59 DanielYang59 mentioned this pull request Oct 22, 2024
Merged
4 tasks
DanielYang59
DanielYang59 commented Oct 26, 2024
View reviewed changes
src/pymatgen/analysis/adsorption.py
@@ -664,6 +662,10 @@ def plot_slab(
decay (float): how the alpha-value decays along the z-axis
inverse (bool): invert z axis to plot opposite surface
"""
# Expensive import (PR4128)
from matplotlib import patches
Copy link
Contributor Author

@DanielYang59 DanielYang59 Oct 26, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

matplotlib is likely not a core module for pymatgen, and incurred noticeable overhead to core.interface:
image

After:
image

@DanielYang59 DanielYang59 changed the title bump monty to use the latest monty.json import speedup patch bump monty to use the latest monty.json import speedup patch, add import test regression test, lazy load some rarely used but costly modules Oct 26, 2024
@DanielYang59 DanielYang59 marked this pull request as ready for review October 26, 2024 10:27
@DanielYang59
Copy link
Contributor Author

DanielYang59 commented Oct 26, 2024 •
edited
Loading

@mkhorton I believe this PR is already for view, let me know if you have any comment, thank you!

Credit to @janosh for the motivation and helpful discussion!

@janosh
Copy link
Member

janosh commented Oct 30, 2024

thanks a lot for your extensive profiling work @DanielYang59! i expected there would be a some low-hanging fruit but not nearly as much as you found. you went above and beyond here. 👍
will be big help in resolving janosh/pymatviz#209

@DanielYang59 DanielYang59 changed the title bump monty to use the latest monty.json import speedup patch, add import test regression test, lazy load some rarely used but costly modules bump monty to use the monty.json import speedup patch, add import time regression test, lazy load some rarely used but costly modules Oct 30, 2024
@shyuep shyuep merged commit 179cdeb into materialsproject:master Nov 13, 2024
43 checks passed
@DanielYang59 DanielYang59 deleted the bump-monty-json branch November 14, 2024 03:31
@DanielYang59
Copy link
Contributor Author

thanks @shyuep this hopefully would significantly speedup import

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@shyuep shyuep Awaiting requested review from shyuep shyuep is a code owner

@mkhorton mkhorton Awaiting requested review from mkhorton mkhorton is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

import monty.json slowing down core import
4 participants
@DanielYang59 @mkhorton @janosh @shyuep